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High-speed cinematography with synchronized oscillography has 
been used to analyze accurately the sound production by a rapidly 
moving structure during communicative behavior of insects (Walker 
& Dew, 1972; Willey, 1973, 1974). However, this method involves 
expensive equipment and perhaps kilometers of film containing only 
a few hundred feet of analyzable sequences. Nevertheless, the ac¬ 
curacy and resolution of audio-visual synchrony by this method has 
been exceeded only by the use of minute magnetometers using the 
Hall-effect (Eisner, 1970; Eisner & Huber, 1973) and by an in¬ 
genious method of flash-photography (Morris & Pipher, 1972). On 
the other hand, studies needing an average synchrony accuracy of 
only ± 5 msec (essentially equivalent to motion picture framing rates 
of 60 pictures per sec) can utilize inexpensive or already available 
video-systems used commonly as educational tools. 

Video has several advantages over cinematographic methods: 
1) sound and picture are routinely recorded synchronously, 2) video¬ 
tape can be erased and reused, 3) some video-cameras have great 
sensitivity and can be used even for nocturnal insects at very low 
light levels, and 4) the system can be used to monitor behavior for 
hours and then sequences can be selected, duplicated on another tape 
and then the original tape can be used again. There are disadvan¬ 
tages of video for high resolution work and these will be discussed 
in the Analysis (q.v.). * . - 
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Fig. 1. Block schematic diagram of the equipment connected for line- 
by-line synchronization of a video field with a simultaneous full sweep of 
the cathode ray gun of the oscilloscope. The CRO is laid on its side so 
that the horizontal sweep passes from top to bottom of the superimposed 
field in VTR 2 (see text, sect. 8, Analysis). For ordinary horizontal sweep, 
the CRO is in normal position and the external trigger on the vertical sync 
is disconnected. 

It seems strange that, with the exception of publications from our 
laboratory (Steinberg & Conant, 1974; Steinberg & Willey, 1974), 
we have found only a few studies of insect behavior which have 
involved the analysis of video-taped sounds and motion (Loher & 
Chandrashekaran, 1970, 1972; Hoeg-Guldberg, 1972). Moreover, 
even these studies have neglected to capitalize on the audio-recording 
and electronic potential of the video-tape system. We have devised a 
simple method to superimpose the oscillographic trace of the sound 
already recorded on any pre-existing video-tape on a duplicate record¬ 
ing. Extensions of this method to synchronous oscillography trans¬ 
duced from any mode will make video monitoring of insect behavior 
amenable to more versatile and precise analysis. 

Method 

A sequence was video-recorded with the following sound-related 
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parameters noted: i) distance from the microphone to the subect, 
and 2) the average temperature of air between subject and micro¬ 
phone. Then we played the recorded sequence on a compatible 
video-tape reproducer (VTR.!) which was linked by a multichannel 
synchronizing circuit (SWITCHER) to another video-recorder 
(VTR 2 ) which preferably used at least one-inch tape. Figure i 
diagrams the set-up schematically. The audio circuit of the VTR X 
was linked also to the input of an oscilloscope (CRO), the screen 
(CRT) of which has an image persistence on the order of micro¬ 
seconds (Pii phosphor coating). A video-camera (CAM) focused 
on the CRT image which was positioned so that it would not mask 
the image of the subject in the superposition. If the trace was to be 
positioned along the bottom of the frame, the horizontal sweep of 
the CRO was set at IO msec/cm so that a single sweep of 12.5 cm 
will encompass at least eight video fields (q. v., sect. 7 & 8, Analysis 
and Fig. 1 for further explanation and an alternative method). We 
also masked the negative potentials with opaque tape. The video¬ 
camera is connected with the VTR 2 by the vertical interval switcher 
and the CRT screen is shielded by an improvised hood from the 
room lights. The results of superposition are simultaneously video- 
monitored. 3 

The resulting video-tape, minus all the inactive time, can be played 
on a VTR which has stop-action and single-field forwarding capabil¬ 
ity (or the tape can be moved forward by hand). The audio signal 
can still be heard as well as seen as an oscillo-trace when the VTR 
is running at normal speed. At low speed the audio circuit auto¬ 
matically turns off and, although an adustment can be made in the 
machine’s circuitry to enable the sound to be heard, with the oscillo¬ 
graphic trace this is neither necessary nor desirable. With the VTR 
stopped on a single field, we marked with a wax crayon the positions 
of the moving parts on the projection screen of the monitor and made 
measurements of the angle changes with a protractor (a better method 
will be outlined later in section 9, Analysis). 

Equipment used: “Quiet Room” (Suttle, Inc., Chicago), Audiometric 
Room, double-walled (Industrial Acoustics, New York), Sony EV-310 25 mm 
videocorder, Scotch Brand MT 20568 video-tape, General Electric model 
4TE33D1 TV camera with a Soligor 10 cm lens, Sony AVC 3200 TV 
camera with Angenieux 7.5 cm lens with a +1 diopter auxiliary lens, 
AKG shotgun microphone model D900, vertical interval switcher-sync 
generator with sync lock capability (Shintron 360), Tektronix 502A os¬ 
cilloscope with Pll phosphor, Panasonic video-monitors (3-inch), one 50W 
electric spot bulb. 
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Analysis 

Analysis of audio-visual synchrony must consider the following 
factors: 

1) A video field is scanned from top of the screen to the bottom 
once in 16.67 msec at a maximum resolution of 262.5 lines per field; 
the horizontal sweep time for a line is 0.0635 msec. The vertical 
blanking interval (retrace time for the electron-gun to move from 
bottom of the screen to the top) requires up to 21 lines, and sub¬ 
tracted from the maximum resolution leaves a net resolution of 241.5 
lines per field with a net duration of 15.5 msec, and a vertical blank¬ 
ing interval of 1 to 1.2 msec. 

2) A video frame consists of two interdigitating video fields with 
a total net duration of 31 msec and a total net resolution of 483 
lines. 

3) Transduction of light and electronic impulses can be regarded 
as instantaneous, however the speed of sound in air is a variable. 
Near sea level, at 37 C and no greater than 10 cm from the micro¬ 
phone, one can calculate that the maximum time interval between 
sound production and electronic transduction is no greater than 0.4 
msec and can be neglected as an important factor. 

4) A crucial discrepancy in visual-acoustic synchrony and visual 
analysis of movements is the time taken by the electron-gun of the 
video camera to trace any action within a field. This factor causes 
three time-related distortions in the image which must be considered 
in the analysis, namely (a) vertical loss of resolution, (b) excessive 
curvature of a rotating image, and (c) inability to record events 
which occur after a scanning trace has passed until the next field scan. 

5) Vertical loss of resolution: the image formed during a single 
field has a maximum vertical resolution of only 241.5 lines per field 
whereas a 16 mm optical-film frame has 50-60 lines of resolution per 
mm (at the center) or about 700 to 800 lines of resolution overall 
(allowing for loss of about 20 lines of resolution at the front and rear 
edge of the frame). On the other hand, the horizontal resolution of 
a good video system probably approaches that of a comparable optical- 
film system. However evaluation of information content of a video 
field and an optical film frame encompassing 15 msec is obscured by 
the next considerations. 

6) Excessive curvature of rotating image: the time required for 
scanning a single field naturally produces a distorted image in any 
movement having lateral components. Plate ia-e illustrates this 
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PI. 1. A portion of a stridulation (chirp) by Chortophaga viridifasciata, 
showing a pulse of sound in five photographs of consecutive fields (a-e). 
The male which is chirping is outlined in (a), (b) and (c) show the right 
femur blurred as it makes the downward stroke (left femur is missing). 
The two vertical black lines drawn on the trace in (d) bracket the portion 
of the oscillotrace which was scanned at the time the leg was moving in 
this field. To the right of the lines is the base of a trace which really was 
much higher, but its peaks will not be scanned until the next field (e). 
The image persistence is due to retention by the video-camera pick-up tube 
and is about equal to one field ( = 16 msec). The image in (f) is the 
result of triggering and synchronizing the CRT sweep with the “front 
porch” of the vertical field (set up as in Fig. 1). The field is identical to 
(e), but only the most intense peaks were registered. (See text, sect. 8) 
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problem with the oscillographic trace and a barely detectable clock¬ 
wise curvature of the femur tip relative to the base in ib and ic. 
This is caused by the fact that a video field is a metachronic series of 
241.5 lines each representing a “shutter speed” of 1/15,750 sec. The 
scanning time of the femur image in PI. la is about 45 lines of 2.86 
msec. In other words, the tip of the femur, which is moving in an 
arc to the left, is imaged nearly 3 msec earlier than that of the femur 
base. Further magnification on the screen would result in greater 
temporal displacements and greater artifact in the shape of the organ. 
Any attempt to measure angles of rotation or points of displacement 
must take into account the video scanning time of the specific image 
and its vector relative to the vertical trace. 

7) Inability to record events after passage of the scanning trace: 
it is usually desirable to superpose the oscillotrace along the bottom 
of the video field (PI. ia-e) where the action is less likely to be 
obscured. However, the lower portion of the field will be scanned 
milliseconds after the action in the center. Nevertheless, visual- 
acoustic synchrony can be computed as shown in PL id. The selec¬ 
tion of the horizontal sweep setting of the CRO is determined by the 
duration of the longest acoustical signal and should be about 40 or 
50 msec longer, so that the oscillotrace of the entire action can be 
visualized in one sweep, and traces in successive frames can be cross- 
referenced. The additional time allows some time before and after 
the signal on the screen so that the beginning or the end of the signal 
and its causative motion aren’t lost in the CRT retrace time, the 
vertical blanking interval, nor at the sometimes distorted edges of the 
monitor screen. Sometimes traces are badly reproduced or oriented, 
but constant inspection of the VTR 2 monitor will allow such mistakes 
to be corrected immediately by duplicating the sequence again. 

8) An alternative method to achieve visualization of motion-sound 
synchrony is shown in Fig. 1 and PI. if. Here the CRO is laid on 
its side so the oscillotrace will sweep from top to bottom of the 
camera pickup tube. The external trigger of the CRT electron-gun 
is triggered on the electronic transition from the vertical sync to the 
start of the first horizontal scanning line of the video field. The hori- 
zonal sweep of the CRT is set to equal the time constant of the video 
field (15 msec per sweep or 1 cm/msec), thus only the sound re¬ 
corded during the video scan will be oscillographed. The oscillotrace 
peaks appearing at the same horizontal plane as the video-recorded 
movement will be relatively synchronous with that movement, with 
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allowance for the time lag induced by the speed of the original sound 
in air (PL if, arrows). 

9) Close examination of CRTs can be a health hazard and the 
thickness of the protective cover can induce parallax error. There¬ 
fore, if exact angle and displacement measurements are desired the 
fields should be photographed with careful focussing on the video 
lines. Such photographs will form a permanent working record of 
key fields of the rather fragile video-tape which can be damaged or 
destroyed by the constant friction of scanning in the stopped position. 

Discussion and Conclusions 

This method allows relatively synchronous visualization of motions 
and the sounds produced by the movements if the actions are rela¬ 
tively slow. For example, it is possible to determine whether the stri- 
dulation by a single up-down motion of a grasshopper’s hind femur 
rubbing against the fore-wing is produced during the up-stroke or 
down-stroke, provided the entire motion has a duration of >60 msec. 
In the case of oedipodine grasshoppers, the chirps produced by males 
during courtship usually are produced by motions which last at least 
60 msec and often 100 msec (Willey, 1974). Since many such sig¬ 
nals are complicated by being segmented into pulses within a single 
up-down motion, normal-speed cinematographic and audiospectro- 
graphic investigations of Oedipodinae (Otte, 1970; Willey & Willey, 

1969) have concluded that the first pulse must be produced on the 
up-stroke. However, high-speed motion analysis (Willey, 1974) 
shows that Arphia sulphurea produces the definitive chirp on the 
downstroke only. A very weak signal was produced on the up-stroke 
but was of such low amplitude that it had never been audiospectro- 
graphed. Video-tapes of Chbnarocephala (Loher & Chandrashekaran, 

1970) and Chortophaga (Steinberg & Willey, 1974) also show that 
stridulations are produced only on the downstroke of the femur in 
these species. However, the mechanism for production of two-pulsed 
chirps must be analyzed by the higher speed methods (Willey, 1974). 

Further refinements of this system can involve direct synchrony of 
oscillotrace with the initial video-tape recording. Also the use of a 
light-emitting diode (LED) and a small high speed digital clock in 
the background, synchronized with each video field, would simplify 
identification of particular fields. However, the method we have 
reported can be used for improving the analysis of already existing 
tapes. 
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Summary 

Oscillographic traces of sounds recorded on videotapes can be 
superimposed easily and synchronously on a duplicate of those tapes. 
The audio recording remains intact in the duplicate and the oscillo- 
trace allows rather precise analysis of slower movements and the 
sounds they produce. This system was tested on pre-existing tapes 
of courtship in the grasshopper Chorbophaga viridifasciata (De 
Geer). The method promises to be useful in analysis of tapes used 
for monitoring behavior and also as a teaching device. 
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